(of companions) To lose contact, to lose one's friendship or closeness.
We used to be good friends but we drifted apart over the years.